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Summary—Man-computer symbiosis is an expected develop- 
ment in cooperative interaction between men and electronic 
computers. It will involve very close coupling between the 
human and the electronic members of the partnership. The main 
aims are 1) to let computers facilitate formulative thinking as 
they now facilitate the solution of formulated problems, and 2) 
to enable men and computers to cooperate in making decisions 
and controlling complex situations without inflexible dependence 
on predetermined programs. In the anticipated symbiotic part- 
nership, men will set the goals, formulate the hypotheses, deter- 
mine the criteria, and perform the evaluations. Computing 
machines will do the routinizable work that must be done to 
prepare the way for insights and decisions in technical and 
scientific thinking. Preliminary analyses indicate that the sym- 
biotic partnership will perform intellectual operations much 
more effectively than man alone can perform them. Prerequisites 
for the achievement of the effective, cooperative association 
include developments in computer time sharing, in memory 
components, in memory organization, in programming lan- 
guages, and in input and output equipment. 


I. INTRODUCTION 
A. Symbiosis 


HE fig tree is pollinated only by the insect Blasto- 

phaga grossorum. The larva of the insect lives in 

the ovary of the fig tree. and there it gets its 
food. The tree and the insect are thus heavily inter- 
dependent: the tree cannot reproduce without the insect; 
the insect cannot eat without the tree; together, they 
consitute not only a viable but a productive and thriving 
partnership. This cooperative “living together in inti- 
mate association, or even close union, of two dissimilar 
organisms” is called symbiosis.? 

“Aan-computer symbiosis” is a subclass of man- 
machine systems.” There are many man-machine sys- 
tems. At present, however, there are no man-computer 
svmbioses. The purposes of this paper are to present 
the concept and, hopefully, to foster the development of 
man-computer symbiosis by analyzing some problems 
of interaction between men and computing machines, 
calling attention to applicable principles of man-machine 
engineering, and pointing out a few questions to which 
research answers are needed. The hope is that, in not 
too many vears, human brains and computing machines 


* Manuscript received by the PGHFE, January 18, 1960; re- 
vised manuscript received, January 18, 1960. The background 
work on which this paper is based was supported largely by the 
Behavioral Sciences Division, Air Force Office of Scientific Re- 
search, Air Research and Development Command, through Con- 
tract No. AF-49(638)-355. 

+ Bolt Beranek and Newman Inc., Cambridge, Mass. 

1“Webster’s New International Dictionary,” 2nd ed., G. and C. 
Merriam Co., Springfield, Mass., p. 2555; 1958. 


will be coupled together very tightly, and that the re- 
sulting partnership will think as no human brain has 
ever thought and process data in a way not approached 
by the information-handling machines we know todav. 


B. Between “Alechanically Extended Man” and 
“Artificial Intelligence” 


As a eoncept, man-computer symbiosis is different. in 
an important way from what North? has called ‘me- 
chanically extended man.”’ In the man-machine systems 
of the past, the human operator supplied the initiative, 
the direction, the integration, and the criterion. The 
mechanical parts of the systems were mere extensions, 
first of the human arm, then of the human eye. These 
systems certainly did not consist of “dissimilar organ- 
isms living together...’ There was only one kind of 
organisin—man—and the rest was there only to help 
him. 

In one sense of course, any man-made system is in- 
tended to help man, to help a man or men outside the 
xvstem. If we foeus upon the human operator(s) within 
the svstem, however, we see that, in some areas of tech- 
nology, a fantastic change has taken place during the 
last few years. “Mechanical extension” has given way 
to replacement of men, to automation, and the men who 
remain are there more to help than to be helped. In 
some instances, particularly in large computer-centered 
information and control systems, the human operators 
are responsible mainly for functions that. 1t proved in- 
feasible to automate. Such systems (“humanly extended 
machines,” North might eall them) are not symbiotic 
systems. They are “semi-automatic” systems, systems 
that started out to be fully automatie but fell short of 
the goal. 

Man-computer symbiosis is probably not the ultimate 
paradigm for complex technological systems. It seems 
entirely possible that, in due course, electronic or chem- 
ical “‘machines” will outdo the human brain in most of 
the functions we now consider exclusively within its 
provinee. Even now, Gelernter’s IBM-704 program for 
proving theorems in plane geometry proceeds at about 
the same pace as Brooklyn high sehool students, and 
makes similar errors.? There are, in fact, several theorem- 
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proving, problem-solving, chess-playing, and pattern- 
recognizing programs (too many for complete refer- 
ence*!®) capable of rivaling human intellectual perform- 
ance in restricted areas; and Newell, Simon, and Shaw’s?® 
“general problem solver” may remove some of the re- 
strictions. In short, it seems worthwhile to avoid argu- 
ment with (other) enthusiasts for artificial intelligence 
by conceding dominance in the distant future of cere- 
bration to machines alone. There will nevertheless be 
a fairly long interim during which the main intellectual 
advances will be made by men and computers working 
together in intimate association. A multidisciplinary 
study group, examining future research and develop- 
ment problems of the Air Force, estimated that it would 
be 1980 before developments in artificial mmtelligence 
make it possible for machines alone to do much thinking 
or problem solving of military significance. That would 
leave, say, five years to develop man-computer symbiosis 
and 15 years to use it. The 15 may be 10 or 500, but those 
vears should be intellectually the most creative and ex- 
citing in the history of mankind. 


II. Aims or MAN-COMPUTER SYMBIOSIS 


Present-day computers are designed primarily to 
solve preformulated problems or to process data accord- 
ing to predetermined procedures. The course of the com- 
putation may be conditional upon results obtained dur- 
ing the computation, but all the alternatives must be 
foreseen in advance. (If an unforeseen alternative arises, 
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the whole process comes to a halt and awaits the neces- 
sary extension of the program.) The requirement for 
preformulation or predetermination is sometimes no 
great disadvantage. It is often said that programming 
for a computing machine forces one to think clearly, that 
it disciplines the thought process. If the user can think 
his problem through in advance, symbiotic association 
with a computing machine is not necessary. 

However, many problems that can be thought through 
in advance are very difficult to think through in ad- 
vance. They would be easier to solve, and they could be 
solved faster, through an intuitively guided trial-and- 
error procedure in which the computer cooperated, turn- 
ing up flaws in the reasoning or revealing unexpected 
turns in the solution. Other problems simply cannot be 
formulated without computing-machine aid. Poincaré 
anticipated the frustration of an important group of 
would-be computer users when he said, “The question 
is not, ‘What is the answer?’ The question is, ‘What is 
the question?’ ”’ One of the main aims of man-computer 
symbiosis is to bring the computing machine effectively 
into the formulative parts of technical problems. 

The other main aim is closely related. It is to bring 
computing machines effectively into processes of think- 
ing that must go on in “real time,” time that moves too 
fast to permit using computers in conventional ways. 
Imagine trying, for example, to direct a battle with the 
aid of a computer on such a schedule as this. You 
formulate your problem today. Tomorrow you spend 
with a programmer. Next week the computer devotes 5 
minutes to assembling your program and 47 seconds to 
calculating the answer to your problem. You get a sheet 
of paper 20 feet long, full of numbers that, instead of 
providing a final solution, only suggest a tactic that 
should be explored by simulation. Obviously, the battle 
would be over before the second step in its planning was 
begun. To think in interaction with a computer in the 
same way that you think with a colleague whose com- 
petence supplements your own will require much tighter 
coupling between man and machine than is suggested by 
the example and than is possible today. 


III. NEED FoR COMPUTER PARTICIPATION IN FORMULATIVE 
AND REAL-TIME THINKING 


The preceding paragraphs tacitly made the assump- 
tion that, if they could be introduced effectively into the 
thought process, the functions that can be performed by 
data-processing machines would improve or facilitate 
thinking and problem solving in an important way. That 
assumption may require justification. 


A. A Preliminary and Informal Time-and-Motion 
Analysis of Technical Thinking 


Despite the fact that there is a voluminous literature 
on thinking and problem solving, including intensive 
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case-history studies of the process of invention, I could 
find nothing comparable to a time-and-motion-study 
analysis of the mental work of a person engaged in a 
scientific or technical enterprise. In the spring and sum- 
mer of 1957, therefore, I tried to keep track of what 
one moderately technical person actually did during the 
hours he regarded as devoted to work. Although I was 
aware of the inadequacy of the sampling, I served as 
my own subject. 

It soon became apparent that the main thing I did 
Was to keep records, and the project would have become 
aun infinite regress if the keeping of records had been 
‘carried through in the detail envisaged in the initial 
plan. It was not. Nevertheless, [ obtained a picture of 
my activities that gave me pausc. Perhaps my spectrum 
is not tvpical—I hope it is not, but I fear it is. 

About 85 per cent of my “thinking” time was spent 
getting Into a position to think, to make a decision, to 
learn something I needed to know. Much more time went 
into finding or obtaining information than into digesting 
it. Hours went into the plotting of graphs, and other 
hours into instrueting an assistant how to plot. When 
the graphs were finished, the relations were obvious at 
once, but the plotting had to be done in order to make 
them so. At one point, it Was necessary to compare six 
expernnental determinations of a funetion relating 
speech-intelligibility to speech-to-noise ratio. No two 
experimenters had used the same definition or measure 
of speech-to-noise ratio. Several hours of ealeulating 
were required to get the data into comparable form. 
When they were in comparable form, it took only a few 
seconds to determine what I needed to know. 

Throughout the period T examined, in short, my 
“thinking” time was devoted mainly to activities that 
were essentially clerical or mechanical: searching, cal- 
culating, plotting, transforming, determining the logical 
or dynamic consequences of a set of assumptions or hy- 
potheses, preparing the way for a decision or an insight. 
Moreover, my choices of what to attempt and what not 
to attempt were determined to an embarrassingly great 
extent by considerations of clerieal feasibility, not in- 
tellectual capability. 

The main suggestion conveyed by the findings just 
deseribed is that the operations that fill most of the 
tune allegedly devoted to technical thinking are opera- 
tions that ean be performed more effectively by ma- 
chines than by men. Severe problems are posed by the 
fact that these operations have to be performed upon 
diverse variables and im unforeseen and continually 
changing sequences. If those problems can be solved in 
such a way as to create a svinbiotie relation between a 
man and a fast information-retrieval and data-process- 
ing machine, however, it seems evident that the coopera- 
tive interaction would greatly improve the thinking 
process. 
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Lb. Comparative Capabilities of Men and Computers 


It may be appropriate to acknowledge, at this point, 
that we are using the term “computer” to cover a wide 
Class of calculating, data-processing, and information- 
storage-and-retrieval machines. The capabilities of ma- 
chines im this elass are increasing almost daily. It. is 
therefore hazardous to make general statements about 
capabilities of the class. Perhaps it is equally hazardous 
to make general statements about the capabilities of 
men. Nevertheless, certain genotypic differences in cap- 
ability between men and computers do stand out. ane 
they have a bearing on the nature of possible man- 
computer svinbiosis and the potential value of achieving 
it. 

As has been said in various ways, men are noisy. nar- 
row-band devices, but their nervous systems have very 
many parallel and simultaneously active channels. Rela- 
tive to men, computing inachines are very fast and very 
wecurate, but thev are constrained to perform only one 
or a few elementary operations at a tune. Men are flexi- 
ble, capable of “programming themselves contingently” 
on the basis of newly received information. Computing 
machines are single-minded, constrained by their “pre- 
programming.” Alen naturally speak redundant. lan- 
guages organized around unitary objects and coherent 
actions and emploving 20 to 60 elementary svimbols. 
Computers “naturally” speak nonredundant languages. 
usually with only two elementary symbols and no inher- 
ent appreciation either of unitary objects or of coherent 
actions. 

To be ngorously correct. those characterizations would 
have to include many qualifiers. Nevertheless, the pic- 
ture of dissimilarity (and therefore potential supplemen- 
tation) that they present is essentially valid. Computing 
machines can do readily, well, and rapidly many things 
that are diffeult or impossible for man, and men ean do 
readily and well, though not rapidly, many things that 
are difficult or impossible for computers. That suggests 
that a svinbiotic cooperation, if suecessful in integrating 
the positive characteristies of men and computers, would 
be of great value. The differences in speed and im lan- 
vuage, of course, pose ciffieulties that must. be overcome. 


ITV. SEPARABLE FUNCTIONS OF MIEN AND COMPUTERS 
IN THE ANTICIPATED SYMBIOTIC ASSOCTATION 


It seems likely that the contributions of human op- 
erators and equipment will blend together so completely 
in many operations that it will be diffieult to separate 
them neatly in analysis. That would be the case if, in 
gathering data on which to base a decision, for example. 
both the man and the computer came up with relevant 
precedents from experience and if the computer then 
suggested a course of action that agreed with the man's 
intuitive judgment. (In theorem-proving programs, com- 
puters find precedents in experience, and in the SAGE 
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System, they suggest courses of action. The foregoing is 
not. a far-fetched example.) In other operations, however, 
the contributions of men and equipment will be to some 
extent separable. 

Men will set the goals and supply the motivations, of 
course, at least in the early years. They will formulate 
hypotheses. They will ask questions. They will think of 
mechanisms, procedures, and models. They will remem- 
ber that such-and-such a person did some possibly rele- 
vant work on a topic of interest back in 1947, or at any 
rate shortly after World War II, and they will have an 
idea in what journals it might have been published. In 
general, they will make approximate and fallible, but 
leading, contributions, and they will define criteria and 
serve as evaluators, judging the contributions of the 
equipment and guiding the general line of thought. 

In addition, men will handle the very-low-probability 
situations when such situations do actually arise. (In 
current man-machine systems, that is one of the human 
operator’s most important functions. The sum of the 
probabilities of very-low-probability alternatives is often 
much too large to neglect.) Men will fill in the gaps, ei- 
ther in the problem solution or in the computer program, 
when the computer has no mode or routine that is ap- 
plicable in a particular circumstance. 

The information-processing equipment, for its part, 
will convert hypotheses into testable models and then 
test the models against data (which the human operator 
may designate roughly and identify as relevant when 
the computer presents them for his approval). The 
equipment will answer questions. It will simulate the 
mechanisms and models, carry out the procedures, and 
display the results to the operator. It will transform 
data, plot graphs (‘‘cutting the cake” in whatever way 
the human operator specifies, or in several alternative 
ways if the human operator is not sure what he wants). 
The equipment will interpolate, extrapolate, and trans- 
form. It will convert static equations or logical state- 
ments into dynamic models so the human operator can 
examine their behavior. In general, it will carry out the 
routinizable, clerical operations that fill the intervals 
between decisions. 

In addition, the computer will serve as a statistical- 
inference, decision-theory, or game-theory machine to 
make elementary evaluations of suggested courses of ac- 
tion whenever there is enough basis to support a formal 
statistical analysis. Finally, it will do as much diagnosis, 
pattern matching, and relevance recognizing as it profit- 
ably can, but it will accept a clearly secondary status in 
those areas. 


V. PREREQUISITES FOR REALIZATION OF 
MAN-COMPUTER SYMBIOSIS 


The data-processing equipment tacitly postulated in 
the preceding section is not available. The computer pro- 
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grams have not been written. There are in fact several 
hurdles that stand between the nonsymbiotic present and 
the anticipated symbiotic future. Let us examine some 
of them to see more clearly what is needed and what the 
chances are of achieving it. 


A. Speed Mismatch Between Alen and Computers 


Any present-day large-scale computer is too fast and 
too costly for real-time cooperative thinking with one 
man. Clearly, for the sake of efficiency and economy, 
the computer must divide its time among many users. 
Time-sharing systems are currently under active devel- 
opment. There are even arrangements to keep users from 
“clobbering” anything but their own personal programs. 

It seems reasonable to envision, for a time 10 or 15 
years hence, a ‘thinking center” that will incorporate the 
functions of present-day libraries together with antici- 
pated advances in information storage and retrieval and 
the symbiotic functions suggested earlier in this paper. 
The picture readily enlarges itself into a network of such 
centers, connected to one another by wide-band com- 
munication lines and to individual users by leased-wire 
services. In such a system, the speed of the computers 
would be balanced, and the cost of the gigantic memories 
and the sophisticated programs would be divided by the 
number of users. 


B. Memory Hardware Requirements 


When we start to think of storing any appreciable 
fraction of a technical literature in computer memory, 
we run into billions of bits and, unless things change 
markedly, billions of dollars. 

The first thing to face is that we shall not store all the 
technical and scientific papers in computer memory. We 
may store the parts that can be summarized most suc- 
cinctly—the quantitative parts and the reference cita- 
tions—but not the whole. Books are among the most 
beautifully engineered, and human-engineered, com- 
ponents in existence, and they will continue to be func- 
tionally important within the context of man-computer 
symbiosis. (Hopefully, the computer will expedite the 
finding, delivering, and returning of books.) 

The second point is that a very important section of 
memory will be permanent: part indelible memory and 
part published memory. The computer will be able to 
write once into indelible memory, and then read back 
indefinitely, but the computer will not be able to erase 
indelible memory. (It may also over-write, turning all 
the 0’s into 1’s, as though marking over what was written 
earlier.) Published memory will be “read-only” memory. 
It will be introduced into the computer already struc- 
tured. The computer will be able to refer to it repeatedly, 
but not to change it. These types of memory will be- 
come more and more important as computers grow larger. 
They can be made more compact than core, thin-film, or 


5 IRE TRANSACTIONS ON HUMAN FACTORS IN ELECTRONICS 


even tape memory, and they will be much less expen- 
sive. The main engineering problems will concern selec- 
tion cireuitry. 

In so far as other aspects of memory requirement are 
concerned, we may count upon the continuing develop- 
ment of ordinary scientific and business computing ma- 
ehines. There is some prospect that memory elements 
will beeome as fast as processing (logic) elements. That 
development would have a revolutionary effeet upon the 
design of computers. 


(. Memory Organization Requirements 


Iinplicit in the idea of man-computer symbiosis are 
the requirements that information be retrievable both 
by name and by pattern and that it be accessible through 
procedure much faster than serial search. At least half 
of the problem of memory organization appears to reside 
in the storage procedure. Most of the remainder seems 
to be wrapped up in the problem of pattern recognition 
within the storage mechanism or medium. Detailed dis- 
cussion of these problems is bevond the present scope. 
However, a brief outline of one promising idea, “trie 
memory,” may serve to indicate the general nature of 
anticipated developments. 

Trie memory is so called by its originator, Fredkin.™ 
because it is designed to facilitate retrieval of informa- 
tion and because the branching storage structure, when 
developed, resembles a tree. Most common memory svs- 
tems store functions of arguments at locations designated 
by the arguments. (In one sense, they do not store the 
arguments at all. In another and more realistic sense, 
they store all the possible arguments in the framework 
structure of the memory.) The tre memory system, on 
the other hand, stores both the functions and the argu- 
ments. The argument is introduced into the memory first. 
one character at a time, starting at a standard initial 
register. Each argument register has one cell for each 
character of the ensemble (e.g., two for information en- 
coded in binary form) and each character cell has within 
it. storage space for the address of the next register. The 
argument is stored by writing a series of addresses, each 
one of which tells where to find the next. At the end of 
the argument is a special ‘‘end-of-argument™ marker. 
Then follow directions to the function, which 1s stored 
in one or another of several ways, either further trie 
structure or “list structure” often being most. effective. 

The trie memory scheme 1s ineficient for small mem- 
ories, but it becomes increasingly efficient in using avail- 
able storage space as memory size increases. The attrac- 
tive features of the scheme are these: 1) The retrieval 
process is extremely simple. Given the argument, enter 
the standard initial register with the first character, and 
pick up the address of the second. Then go to the second 
register, and pick up the address of the third, ete. 2) Tf 
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two arguments have initial characters in common, they 
use the same storage space for those characters. 3) The 
leneths of the arguments necd not be the same, and need 
not be specified in advanee. 4) No room in storage is 
reserved for or used by any argument until it is aetually 
stored. The tric structure is created as the items are m- 
troduced into the memory. 5) A funetion ean be used as 
an argument for another function, and that funetion as 
an argument for the next. Thus. for example, by enter- 
ing with the argument, “matrix multipheation.” one 
nught retrieve the entire program for performing 2 ma- 
trix mmultipheation on the eomputer. 6) By examining 
the storage at a given level, one can determine What thus- 
far similar items have been stored. For example, if there 
ix no citation for Mean, J.P. itis but a step or two hack- 
ward to pick up the trail of Egan, James. ... 

The properties just deseribed do not inelude all the 
desired ones, but they bring computer storage into res- 
onanee with human operators and them predilection to 
designate things by naming or pointing. 


D. The Language Problem 


The basie dissinuliarity between human languages and 
computer languages may be the most serious obstacle to 
true symbiosis. It is reassuring, however, to note what 
great strides have already been made, through interpre- 
tive programs and particularly through assembly or com- 
piling programs such as FORTRAN, to adapt computers 
to human language forms. The “Information Processing 
Language” of Shaw, Newell, Simon, and Eljis'® represents 
another line of rapprochement. And, in ALGOL and re- 
lated svstems, men are proving their flexibility by adopt- 
ing standard formulas of representation and expression 
that are readily translatable into machine language. 

For the purposes of real-time cooperation between 
men and computers, it will be necessary, however, to 
make use of an additional and rather different principle 
of communication and control. The idea may be high- 
lighted by comparing instructions ordinarily addressed 
to intelligent human beings with instructions ordinarily 
used with computers. The latter specify precisely the in- 
dividual steps to take and the sequence in which to take 
them. The former present or imply something about in- 
eentive or motivation, and they supply a criterion by 
which the human executor of the instructions will know 
when he has accomplished his task. In short: instrue- 
tions directed to computers specify courses; Imstruc- 
tions directed to human beings specify goals. 

Men appear to think more naturally and easily in 
terms of goals than in terms of courses. True, they usu- 
ally know something about directions in which to travel 
or lines along which to work, but few start out with pre- 
cisely formulated itineraries. Who, for example. would 
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depart from Boston for Los Angeles with a detailed 
specification of the route? Instead, to paraphrase Wiener, 
men bound for Los Angeles try continually to decrease 
the amount by which they are not yet in the smog. 

Computer instruction through specification of goals is 
being approached along two paths. The first involves 
problem-solving, hill-climbing, self-organizing programs. 
The second involves real-time concatenation of prepro- 
grammed segments and closed subroutines which the hu- 
man operator can designate and call into action simply 
by name. 

Along the first of these paths, there has been promis- 
ing exploratory work. It is clear that, working within the 
loose constraints of predetermined strategies, computers 
will in due course be able to devise and simplify their 
own procedures for achieving stated goals. Thus far, the 
achievements have not been substantively important; 
they have constituted only “demonstration in principle.” 
Nevertheless, the implications are far-reaching. 

Although the second path is simpler and apparently 
capable of earlier realization, it has been relatively neg- 
lected. Fredkin’s trie memory provides a promising para- 
digm. We may in due course see a serious effort to de- 
velop computer programs that can be connected together 
like the words and phrases of speech to do whatever 
computation or control is required at the moment. The 
consideration that holds back such an effort, apparently, 
is that the effort would produce nothing that would be of 
great value in the context. of existing computers. It would 
be unrewarding to develop the language before there are 
any computing machines capable of responding mean- 
ingfully to it. 


E. Input and Output Equipment 


The department of data processing that seems least ad- 
vanced, in so far as the requirements of man-computer 
symbiosis are concerned, is the one that deals with input 
and output equipment or, as it is seen from the human 
operator’s point of view, displays and controls. Immedi- 
ately after saving that, it is essential to make qualifying 
comments, because the engineering of equipment for high- 
speed introduction and extraction of information has been 
excellent, and because some very sophisticated display 
and control techniques have been developed in such re- 
search laboratories as the Lincoln Laboratory. By and 
large, in generally available computers, however, there 
is almost no provision for any more effective, immediate 
man-machine communication than ean be achieved with 
an electric typewriter. 

Displays seem to be in a somewhat better state than 
controls. Many computers plot graphs on oscilloscope 
screens, and a few take advantage of the remarkable 
capabilities, graphical and symbolic, of the charactron 
display tube. Nowhere, to my knowledge, however, is 
there anything approaching the flexibility and conven- 
ience of the pencil and doodle pad or the chalk and 
blackboard used by men in technical discussion. 
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1) Desk-Surface Display and Control: Certainly, for 
effective man-computer interaction, it will be necessary 
for the man and the computer to draw graphs and pic- 
tures and to write notes and equations to each other on 
the same display surface. The man should be able to 
present a function to the eomputer, in a rough but rapid 
fashion, by drawing a graph. The computer should read 
the man’s writing, perhaps on the condition that it be in 
clear block capitals, and it should immediately post, at 
the location of each hand-drawn symbol, the correspond- 
ing character as interpreted and put into precise type- 
face. With such an input-output device, the operator 
would quickly learn to write or print in a manner legible 
to the machine. He could compose instructions and sub- 
routines, set them into proper format, and check them 
over before introducing them finally into the computer’s 
main memory. He could even define new symbols, as 
Gilmore and Savell!® have done at the Lincoln Labora- 
tory, and present them directly to the computer. He 
could sketch out the format of a table roughly and let 
the computer shape it up with precision. He could correct 
the computer’s data, instruct the machine via flow dia- 
grams, and in general interact with it very much as he 
would with another engineer, except that the “other en- 
gineer” would be a precise draftsman, a lightning calcu- 
lator, a mnemonic wizard, and many other valuable part- 
ners all in one. 


2) Computer-Posted Wall Display: In some techno- 
logical systems, several men share responsibility for con- 
trolling vehicles whose behaviors interact. Some informa- 
tion must be presented simultaneously to all the men, 
preferably on a common grid, to coordinate their actions. 
Other information is of relevance only to one or two op- 
erators. There would be only a confusion of uninterpret- 
able clutter if all the information were presented on one 
display to all of them. The information must be posted 
by a computer, since manual plotting is too slow to keep 
it up to date. 

The problem just outlined is even now a critical one, 
and it seems certain to become more and more critical 
as time goes by. Several designers are convinced that 
displays with the desired characteristics can be con- 
structed with the aid of flashing lights and time-sharing 
viewing screens based on the light-valve principle. 

The large display should be supplemented, according to 
most of those who have thought about the problem, by 
individual display-control units. The latter would permit 
the operators to modify the wall display without leaving 
their locations. For some purposes, it would be desirable 
for the operators to be able to communicate with the 
computer through the supplementary displays and per- 
haps even through the wall display. At least one scheme 
for providing such communication seems feasible. 


19 J. T. Gilmore and R. E. Savell, “The Lincoln Writer,” Lin- 
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1959. 


10 IRE TRANSACTIONS ON HUMAN FACTORS IN ELECTRONICS 


The large wall display and its associated system are 
relevant, of course, to symbiotic cooperation between a 
computer and a team of men. Laboratory experiments 
have indicated repeatedly that informal, parallel ar- 
rangements of operators, coordinating their activities 
through reference to a large situation display, have im- 
portant advantages over the arrangement, more widely 
used, that locates the operators at) individual consoles 
and attempts to correlate their actions through the agency 
of a computer. This is one of several operator-team prob- 
lenis in need of careful study. 

3) Automatic Speech Production and Recognition: 
How desirable and how feasible is speech communication 
between human operators and computing machines? That 
compound question is asked whenever sophisticated data- 
processing systems are discussed. Engineers who work 
and live with eomputers take a conservative attitude to- 
ward the desirability. Engineers who have had experi- 
ence in the field of automatic speech recognition take a 
conservative attitude toward the feasibilitv. Yet there is 
continuing interest in the idea of talking with computing 
machines. In large part, the interest stems from realiza- 
tion that one can hardly take a military commander or a 
corporation president away from his work to teach him 
to type. If computing machines are ever to be used di- 
reetly by top-level decision makers, it may be worth- 
while to provide communication via the most natural 
means, even at considerable cost. 

Preliminary analysis of his problems and time scales 
suggests that a corporation president would be interested 
in a symbiotic association with a computer only as an 
avocation. Business situations usually move slowly 
enough that there is time for briefings and conferences. 
It seems reasonable, therefore, for computer specialists 
to be the ones who interact. directly with computers in 
business offices. 

The military commander, on the other hand, faces a 
greater probability of having to make critical decisions 
in short intervals of time. It is easy to overdramatize the 
notion of the ten-minute war, but it would be dangerous 
to count on having more than ten minutes in which to 
make a eritical decision. As military system ground en- 
vironments and control centers grow in capability and 
complexity, therefore, a real requirement for automatic 
speech production and recognition In computers seems 
likely to develop. Certainly, if the equipment were al- 
ready developed, reliable, and available, it would be used. 

In so far as feasibility is concerned, speech produc- 
tion poses less severe problems of a technical nature than 
does automatic recognition of speech sounds. A commer- 
cial electronic digital voltmeter now reads aloud its indi- 
cations, digit by digit. For eight or ten vears, at the Bell 
Telephone Laboratories. the Roval Institute of Technol- 
ogy (Stockholm), the Signals Research and Development 
Establishment (Christchurch), the Haskins Laboratory, 
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and the Massachusetts Institute of Technology, Dunn?" 
Fant,?! Lawrence,*?, Cooper" Stevens? and their co- 
workers, have demonstrated successive generations of in- 
telligible automatic talkers. Reeent work at the Haskins 
Laboratory has led to the development of a digital code. 
suitable for use by computing machines, that makes an 
automatic voice utter intelhgible connected discourse.?* 

The feasibility of automatic speech recognition de- 
pends heavily upon the size of the vocabulary of words 
to be recognized and upon the diversity of talkers and 
accents with which it must work. Ninety-eight per cent 
correct recognition of naturally spoken deeimal digits 
Was demonstrated several vears ago at the Bell Tele- 
phone Laboratories and at the Lincoln Laboratory." > 
To go a step up the seale of voeabulary size, we may sav 
that an automatic recognizer of clearly spoken ilpha- 
numerical characters can almost surely be developed now 
on the basis of existing knowledge. Smee untraimed op- 
erators ean read at least as rapidly as trained ones can 
tvpe. sueh a device would be a convenient tool in al- 
most any computer installation, 

For real-time interaction on a truly symbiotic level, 
however, a voeabulary of about 2000 words, e.y., 1000 
words of something like basie English and 1000 technical 
terms, would probably be required. That constitutes a 
challenging problem. In the consensus of acoustielns 
and linguists, construction of a recognizer of 2000 words 
eannot he accomplished now. However, there are several 
organizations that would happily undertake to develop 
and automatic recognizer for sueh a vocabulary on a five- 
vear basis. They would stipulate that. the speech be elear 
speech, dictation style, without unusual aecent. 

Although detailed diseussion of techniques of auto- 
matie speech recognition is beyond the present scope, it 
is fitting to note that eomputing machines are playing 
a dominant role in the development of automatic speech 
recognizers. They have contributed the impetus that 
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accounts for the present optimism, or rather for the opti- 
mism presently found in some quarters. Two or three 
years ago, it appeared that automatic recognition of size- 
able vocabularies would not be achieved for ten or fifteen 
years; that 1t would have to await much further, gradual 
accumulation of knowledge of acoustic, phonetic, linguis- 
tic, and psychological processes in speech communica- 
tion. Now, however, many see a prospect of accelerating 
the acquisition of that knowledge with the aid of com- 
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puter processing of speech signals, and not a few work- 
ers have the feeling that sophisticated computer pro- 
grams will be able to perform well as speech-pattern 
recognizers even without the aid of much substantive 
knowledge of speech signals and processes. Putting those 
two considerations together brings the estimate of the 
time required to achieve practically significant speech 
recognition down to perhaps five years, the five years 
just mentioned. 


Pattern Recognition and Display Characteristics” 
W. R. BUSHT, R. B. KELLY, anp V. M. DONAHUEF 


Summary—This paper reports experimental results of human 
operator performance in a visual recognition task. The work 
began with a method of generating families of complex patterns 
to simulate certain characteristics of visual sensor displays, 
such as radar and infrared returns. The experimental effort 
was directed toward establishing criteria for predicting human 
operator performance in a map-matching task. The operators’ 
task was to recognize which of four patterns presented simul- 
taneously with a reference pattern belonged to the reference 
pattern family. The measure of performance was the time in 
seconds taken by the operator to make a selection. Response 
times were more rapid when the reference pattern was less 
complex than the comparison than when the reference pattern 
was the more complex, Analysis of the display characteristics 
led to the selection of four physical measures to be used in pre- 
dicting operator performance. These measures—pattern length, 
pattern density, and two measures of pattern complexity—cor- 
related highly with response time, were not highly intercorre- 
lated, and were applicable to natural sensor returns. The four 
measures were found to account for a high degree of the total 
variance. Regression equations were derived which predict per- 
formance from known values of the four measures. 


INTRODUCTION 


ECENT developments in high-speed, high-alti- 
tude aircraft have imposed increased demands 
on the visual performance of the human operator. 

Typical operational tasks include the monitoring of com- 
plex navigational, target and predictive displays. A rep- 
resentative problem encountered in the study of visual 
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performance under field conditions is that of “map- 
matching,” by which radar operators orient with the 
ground. This task involves the comparison of two com- 
plex pattern displays in order to determine whether a 
reference pattern is contained within a comparison. If 
performance is to be predicted and improved in this sit- 
uation, research must be conducted to identify the rele- 
vant variables and any interactions which may exist 
among them. 

Hake! has discussed a number of the factors influenc- 
ing pattern vision. Among these are the amount or den- 
sity of information contained in the display, time re- 
quirements, contrast between component parts, adapta- 
tion, and a number of operator conditions, including 
training and expectancies. Little past research relates 
directly to the field conditions discussed, either in terms 
of time restrictions or of the type and complexity of the 
stimulus materials. As Hake points out, results depend 
to a considerable extent upon specific methodology. This 
fact would seem to justify a close correspondence be- 
tween experimental conditions and that particular situa- 
tion about which recommendations are to be made. 

Boynton and Bush,? using various sorts of geometric 
forms, have determined that the probability of recogni- 
tion of a critical target embedded within other forms de- 
creases linearly with the logarithm of the number of con- 
fusion forms. It is predicted from these data that per- 
formance will differ depending on the number of elements 
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